Accounting for Multiple Comparisons in Decision Tree Pruning

Overtting is a widely observed pathology of induction algorithms. For induction algorithms that build decision trees, pruning is a common approach to correct overtting. Most common pruning techniques , do not account for one potentially important factor | multiple comparisons. Multiple comparisons occur whenever an induction algorithm examines several candidate models and selects the one that best accords with the data. Making multiple comparisons produces systematic overestimates of accuracy. This paper empirically examines the importance of accounting for multiple comparisons when evaluating models. Specically, it examines the eectiveness of one particular pruning method that does account f o r m ultiple comparisons { Bonferroni pruning. Based on experiments with articial and realistic datasets, Bonferroni pruning produces trees that are smaller and at least as accurate as trees pruned using several other common approaches. pruning Abstract Accounting for Multiple Comparisons in Decision Tree Pruning 1 Introduction